
Advances in Engineering Education 



An Exploration of the Effectiveness of Product Archeol¬ 
ogy in an Undergraduate Engineering Curriculum: What 
can a Five-Hour Curriculum Do? 

GUL E. OKUDAN KREMER 
TIMOTHY W. SIMPSON 
AND 

OMAR M. ASHOUR 

The Pennsylvania State University 

University Park, PA 


ABSTRACT 


In this paper, we present our efforts in embedding product archeology inspired curricula into two 
engineering courses along with assessment results. The assessment focuses on the effectiveness of 
the embedded curricula in enhancing students’ understanding on the global, societal, environmental, 
and economic (GSEE) implications of engineering design. The results show the significant positive 
impact of the curricula as perceived by students in comparison to other engineering courses, and 
thus, is another testament to the fact that carefully crafted curricula can have substantial impact. 
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INTRODUCTION 

Many scholars have written about the rigidity of engineering curriculum being an inhibiting 
factor to introduce new content or topics. Odds are especially against topics that are considered 
to be “soft”, i.e., not directly meant to increase the analytical capability of engineering students. 
Given this setting, it is critical that we find pedagogies that are effective and efficient in particular 
for non-analytic, yet important and practical topics that foster professional development. In this 
paper, we present a study of the effectiveness of product archeology as pedagogy to introduce 
and enhance student awareness of global, societal, economic and environmental (GSEE) issues in 
an undergraduate engineering curriculum. 
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Product archeology is a new pedagogical paradigm that embeds context into product dissec- 
tion activities. Product dissection has been very successful when used in engineering courses to 
help anchor the knowledge and practice of engineering in students’ minds; however, most product 
dissection activities tend to stress form, function, and fabrication, missing opportunities to explore 
the broader impacts of engineering design decisions. Contextualization brought through product 
archeology affords opportunities to seamlessly blend the GSEE concepts to student learning in an 
active, explorative manner. 

In order to understand the effectiveness of this new pedagogical paradigm, we embedded it in 
two different courses at Penn State (EDSGN 100 and IE 466) and collected data pertaining to student 
perceptions using an established survey instrument. One of the courses, Introduction to Engineer¬ 
ing Design (EDSGN 100), is a required first year course, while the other, Concurrent Engineering (IE 
466), is a senior level technical elective. In this paper, we present an overview of our implementa¬ 
tions in these courses along with assessment results. Before proceeding with the overview, we first 
review the relevant literature. 


BACKGROUND 

Contemporary issues induced by globalization, economic turmoil, environmental resource limita¬ 
tions are impacting the way we live in all corners of the world, and hence, more than ever before, 
engineers are required to have a much broader perspective of their profession. The Accreditation 
Board for Engineering and Technology (ABET) has tried to address this in the “a-k outcomes” that 
are now part of EC 2000 (ABET, 1999). However, as departments undergo their second wave of 
accreditation reviews under the new criteria, many are still struggling to satisfy Outcome h, namely, 
“The broad education necessary to understand the impact of engineering solutions in global, eco¬ 
nomic, environmental, and societal context.” For instance, at Trinity College, a first-year design 
course is used to assess every ABET outcome except Outcomes h and i (Ahlgren, 2001). At Purdue, 
involvement in extracurricular activities were used to assess each of the ABET outcomes; however, 
the authors of the study were not able to make any conclusions for Outcomes h, noting the need 
for “further analysis” of this outcome (Dalrymple and Evangelou, 2006). Briedis (2002) noted that 
the assessment of Outcome h was “less straightforward” than the other professional outcomes, and 
a new course had to be developed to address this outcome directly. This strategy was used instead 
of the alternative of using the humanities and social science general education courses to assess 
Outcome h. Unfortunately, most departments neither have the flexibility nor the room to develop 
a new course specifically to address any single ABET outcome, much less Outcome h. As a result, 
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most departments relegate this requirement to their senior capstone design experience along with 
many other outcomes (Biney, 2007), and the end result mostly is an ineffective “catch all” course 
with limited exposure to these increasingly important topics - too little, too late. 

In an effort to respond to Outcome h in a more effective way, we have proposed the use of a 
new “product archeology” paradigm, which prompts students to peel the layers of complexity in 
an engineered solution (product, process, or system), and “dig” the surrounding information and 
artifact layers to understand the historical and contemporary GSEE implications (Simpson et al., 
2011). If we consider consumer products as the artifacts under investigation, this “digging” will entail 
reconstructing the lifecycle of a product - the customer requirements, design specifications, and 
manufacturing processes used to produce it - to understand the decisions that led to its develop¬ 
ment. Indeed, “archaeologists try to reconstruct life and culture of past ages through the study of 
objects created by humans, known as artifacts” (McMillon, 1991). While archaeologists use a variety 
of tools and methods in their work, their approach to a new site can be generalized into four phases 
(Renfrew and Bhan, 2004): (1) preparation, (2) excavation, (3) evaluation, and (4) explanation. These 
phases constitute a useful framework with which we can engage the student beyond the product 
itself, with a deeper look into the context and complexities. 

Figure 1 shows how we use Kolb’s (1984) model of experiential learning to anchor our framework 
in formal learning theory. Specifically, we have mapped Kolb’s four stages of (1) concrete experience, 
(2) reflective observation, (3) abstract conceptualization, and (4) active experimentation to the cor¬ 
responding four phases of archaeological exploration. As shown in the figure, students are asked 
to reflect on what they know about the factors that impact the design of particular products and 


Archaeological Approach to a Site Kolb’s 4-Stage Learning Model 


1. Preparation - >■ 

• Survey the site 

• Gather tools, etc. 

• Historical research 

2. Excavation - >■ 

• Dig and extract 

• Collect specimens 

3. Evaluation - >■ 

• Identify available technology 

• Carbon dating/chronology 

• Analyze found artifacts, food, tools, art, etc. 

4. Explanation - >■ 

• Draw conclusions based on gathered evidence 


1. Reflective Observation 

• Conduct product research 

• Plan dissection process 

• Investigate product lifecycle 

2. Concrete Experience 

• Dissect the product 

• Reverse engineering 

3. Active Experimentation 

• Ask “what if’ type questions 

• Benchmark other products 

• Conduct product and material experiments 

4. Abstract Conceptualization 

• Draw conclusions based on gathered 
evidence 


Figure 1. Mapping between archaeological exploration phases and 
Kolb's mode! (adopted from Lewis et al. 2011). 
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postulate responses to several questions relating to economic, societal, etc. aspects of the designs in 
the preparation phase. The excavation activities serve as concrete experiences where students can 
physically dissect products and perform appropriate research to develop well-reasoned answers 
to specific design-related questions. The evaluation and explanation phases provide opportunities 
for students to actively experiment and abstract meaning from both their research and concrete 
dissection experiences and understand their work in the context of how GSEE factors influence 
design decisions. 

The concept of product archaeology is not new; it was first introduced by Ulrich and Pearson 
(1998) as a way to measure the design attributes that drive cost through analysis of the physical 
products themselves. Analysis of the physical products is mostly done as product dissection activi¬ 
ties within the engineering curriculum and has been in place for two decades. Many forms of such 
activities that are in use today have their roots in Professor Sherri Sheppard’s Mechanical Dissection 
course at Stanford (Sheppard, 1992a and 1992b). Numerous engineering courses (e.g., Beaudoin 
and Ollis, 1995; Lamancusa et al., 1996; Carlson et al., 1997; Sakamoto et aL, 1999; Marchese et aL, 
2003; and Ogot et al., 2008) have drawn upon the materials and activities developed for her course. 
These initial developments were in response to a general agreement by U.S. industry, engineering 
societies, and the federal government that there had been a decline in the quality of undergradu¬ 
ate engineering education over the previous two decades (Fincher, 1986; Nicolai, 1995). As a result, 
there was a push towards providing both intellectual and physical activities (such as dissection) to 
anchor the knowledge and practice of engineering in the minds of students (Lamancusa et al., 1996; 
Brereton, 1998). Product dissection has since become a popular pedagogy for engaging engineering 
students given its “hands-on” nature. Product dissection introduces students to functional products 
and processes, and providing such experiences early in the students’ academic careers increases 
motivation and retention (Carlson et al., 1997). Product dissection can also be used to increase 
awareness of the design process (Otto and Wood, 2001), and such “learning by doing” activities 
encourage the development of curiosity, proficiency, and manual dexterity—three desirable traits 
of an engineer (Beaudoin and Ollis, 1995). 

With the product archeology paradigm we intended to not only have our students experience the 
benefits of product dissection but also have them become keenly aware of the GSEE implications. 
Thus, our view is much broader in the sense that product archaeology provides an opportunity to 
study not only the manufacturing cost (i.e., economic issues) of a product, but also the global and 
societal context that influenced its development. It also provides a context for studying the environ¬ 
mental impact of a product by considering, for example, the energy and material usage throughout 
the life cycle of the product (i.e., from cradle to cradle (McDonough and Braungart, 2002)). When 
implemented in an engineering classroom, product archaeology can allow students to visualize 
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themselves as the designers and, in the time frame during which a specific product is developed, 
to try to re-create the global and local conditions that led to its development. We have created two 
types of exercises wherein students “dig” to uncover not only the manufacturing (i.e., economic) 
issues of a product, but also the global and societal context that influenced its development as well 
as the environmental impact of the product during its life cycle. As introduced in (Simpson et a I., 
2011), a competitive “dig” pits teams of students against each other in a time-based competition to 
unearth the GSEE impact of a product while a collaborative “dig” allows students to work together 
to dig more deeply into these issues. A collaborative “dig” that is used in a first-year and a senior- 
level course is discussed next. 


PRODUCT ARCHAEOLOGY CURRICULUM 

Based on the previous discussion, we have prepared and implemented mini-curricula on product 
archeology in two different courses (EDSGN 100 and IE 466). We introduce these courses next along 
with specific implementation details for this study. 

“Dig” in EDSGN 100: 

Introduction to Engineering Design (EDSGN 100) is a required first-year engineering course for 
all engineering majors. In 2004, the course had major a revision to include product dissection and 
redesign activities as the first project, to be followed by an industry-sponsored second project. For 
the first project, coffee-makers and electric toothbrushes are dissected, focusing on which energy 
analysis and product teardown are done in a structured fashion. Based on these activities, and ac¬ 
companying course lectures on engineering design process, students are then asked to redesign 
the dissected product. 

Given the potential we have articulated in Section 2, we have introduced a product archeology 
curriculum as part of the design project one in Fall 2010. This new curriculum introduced the concept 
of product archeology in addition to product dissection. We share the following definition with our 
students during class implementations. 

We define product archaeology as the process of reconstructing the lifecycle of a product - 
the customer requirements, design specifications, and manufacturing processes used to 
produce it - to understand the decisions that led to its development. Product archaeology 
provides an opportunity to study not only the manufacturing (i.e., economic) issues of a 
product, but also the global and societal context that influenced its development, it also 
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provides a context for studying the environmental impact of a product by considering, for 
example, the energy and material usage throughout the life cycle of the product (i.e., from 
cradle to grave). Through product archaeology, students try to place themselves in the 
minds of designers and in the time frame during which a specific product is developed to try 
to re-create the global and local conditions that led to its development. 

Given this definition, and the context of the project one, we asked students to redesign the dis¬ 
sected coffee-maker for a contemporary Japanese household. As part of the “dig”, students were 
asked to consider the following: 

1. Global: Are coffee-makers common kitchen staples in Japan? If not, what is the equivalent 
drink to coffee in Japan? How do the kitchens change in size across the countries, and why? 

2. Societal: What impact has coffee had on Japanese society? Examine the trends of coffee drink¬ 
ers in Japan over the past decade and compare them to the trends over the same time period 
in the U.S. How do cultural and societal differences in these two countries drive the design of 
the coffee maker? 

3. Economic: How much does a cup of coffee cost? Make sure to factor in the cost of electricity, 
water, filters, and coffee grounds as well as the cost of the coffee maker itself. How much a 
Japanese household might invest in a coffee-maker? What are the added features or function¬ 
alities in coffee-makers? How do these impact the cost of designs? 

A. Environmental: What is the environmental impact of your coffee maker? Again, how much water, 
electricity, coffee, and filter material are you using to make each cup of coffee? What impact 
does that have on the environment (e.g., container, sleeve, milk/cream, sugar)? Is it more (or 
less) than making coffee daily at home? What about the coffee maker itself: can any of it be 
recycled, reused, or remanufactured after it has served its useful life? 

The total duration of project one is six weeks, during which two class sessions (~ 3.5 hours) are devoted 
to product dissection activities. One additional class session was used to introduce the product archeol¬ 
ogy concept (-1.75 hours), but the overall project duration remained unchanged. Students reported on 
the product dissection, archeology, and redesign activities at the end of the project duration. 

“Dig” in IE 466: 

We implemented a collaborative “dig” during both Fall 2010 and Fall 2011 in IE 466. This “dig” was 
part of a senior technical elective on concurrent engineering for students predominantly in industrial 
engineering and mechanical engineering. The course is also one of the three required courses for the 
Product Realization Minor and is intended as a follow-on to the sophomore-level product dissection 
course (within the mechanical engineering curriculum); however, the product dissection course is 
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not a pre-requisite for this course. The instructions for the dig provided direction to students in 
terms of assessing the global, societal, economic, and environmental impact that we wanted them 
to explore in the products they have analyzed. 

As part of the course, teams of 3-4 students each were dissecting and analyzing a specific coffee- 
maker while also applying tools discussed in the course (i.e., House of Quality, Failure Mode Effects 
Analysis, and Design for Manufacturing and Assembly). To complement these regular course activities, 
a collaborative “dig” was introduced to explore specific aspects of the global, societal, economic, 
and environmental impact of their coffee-maker. Specifically, the “dig” assignment required each 
team to assess their coffee maker from the following perspectives: 

1. Global: Where does coffee come from? Investigate the global supply chain for coffee, tracing 
the origin of coffee from where it is grown (you can pick one of the many countries that supply 
coffee) to where you purchase coffee for use in your coffee maker. Also, what can you discern 
about the origin of the coffee maker itself? Where was it made? Were all of the parts fabricated 
in the same place? 

2 .Societal: What impact has coffee had on our society? Examine the trends of coffee drinkers 
in the U.S. over the past decade and compare them to the trends over the same time period 
in one other country of your choosing. How do cultural and societal differences in these two 
countries drive the design of the coffee maker? Consider things like frequency of use, location 
of use, strength of coffee, amount of coffee, etc. 

3. Economic: How much does a cup of coffee cost? Perform an economic analysis on your coffee 
maker. Make sure to factor in the cost of electricity, water, filters, and coffee grounds as well as 
the cost of the coffee maker itself, amortized over its expected life. Are there any other costs 
associated with making a cup of coffee at home? How does this compare to buying a cup of 
coffee at Dunkin Donuts or Starbucks? 

A.Environmental: What is the environmental impact of your coffee maker? Again, how much 
water, electricity, coffee, and filter material are you using to make each cup of coffee? How 
does this compare to buying a cup of coffee each day at Dunkin Donuts or Starbucks? What 
impact does that have on the environment (e.g., container, sleeve, milk/cream, sugar)? Is it 
more (or less) than making coffee daily at home? What about the coffee maker itself: can any 
of it be recycled, reused, or remanufactured after it has served its useful life? Is there a market 
for refurbished coffee makers? Why or why not? 

Students used online resources to investigate many of these questions and gather relevant data 
for the analysis. Teams also relied on their peers in the class to gain initial insight into the global and 
societal aspects of coffee makers and coffee drinking—roughly 10% of the students in the course 
were international. Team reports were prepared in Microsoft Word and submitted to the instructor 
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for grading, and presentations were developed in Microsoft PowerPoint so that results could be 
shared with classmates. As an in-class activity, introduction of the product archeology concept took 
about 75 minutes to discuss, and 75 minutes to present by students. Students had nearly five weeks 
to complete the assignment, i.e., from the time it was introduced to the time their report was due. 
Across Fall 2010 and Fall 2011, there was no substantial difference in the way this curriculum was 
introduced, and the same instructor taught the course. 


EFFECTIVENESS ASSESSMENT METHODOLOGY 

In order to understand the effectiveness of this pedagogical paradigm as we have embedded 
them in two different courses (EDSGN 100 and IE 466) as described in Section 3, we collected data 
pertaining to student perceptions. Because the faculty members who introduced product archeol¬ 
ogy in class also contributed to its development, we opted to collect data using a survey (E2020 
survey) developed for more comprehensive purposes in order to eliminate potential bias (E2020 
survey, http://www.ed.psu.edu/ educ/e2020/surveys-1 ). As summarized in Table 1, the survey asks 
students to evaluate the emphasis and impact that all their engineering courses have had on ethi¬ 
cal issues, life-long learning, beliefs/values, cultural diversity, creativity, workforce trends, emerging 
technologies, practical use of theories, professional skills, communication skills, leadership skills, 
working in teams, project management, cultural context of engineering solutions, the impact of non¬ 
engineering fields, systems thinking, the application of knowledge from other fields to solve engi¬ 
neering problems, defining a design problem, and generating solutions to an engineering problem. 
The E2020 survey was tailored to evaluate the emphasis and impact of the product archaeology 
curriculum on the same set of issues. Specific questions included, for example, “Overall, how much 
have all the courses you’ve taken in your engineering program collectively emphasized each of the 
following Topics in Engineering (Professional Skills, Problem Solving Skills).” Note that scales 10 
and 13,11 and 14, and 12 and 15 are designed to be evaluated in pairs. The first in each pair (i.e., 10, 
11, 12) asks about the relevant items as they relate to students’ all engineering courses, while the 
latter highlights a specific course. This paired system of scales provides a mechanism to evaluate a 
course’s impact on selected issues in comparison to all others’. 

For EDGSN 100 students, the survey was administered during Fall 2010 semester; for IE 466 
students the same data collection was done in Fall 2010 and then in Fall 2011 in order to collect 
sufficient data for statistical analysis. The initial data analysis for the IE 466 implementation was 
reported earlier (Simpson et al., 2011); however, since number of participants who volunteered to 
partake in the study was only 18, we have repeated the data collection during Fall 2011. 
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# 

Scale 

Scale 1 

Applying Math and Science 

Scale 2 

Defining Problems and Generating Design Solutions 

Scale 3 

Managing a Design Project 

Scale 4 

Engineering Context 

Scale 5 

Teamwork 

Scale 6 

Communication 

Scale 7 

Leadership 

Scale 8 

Interdisciplinary Knowledge and Skills 

Scale 9 

Recognizing Perspectives 

Scale 10 

The Emphasis of all Taken Courses in Engineering on Some Topics in Engineering 

Scale 11 

The Emphasis of all Taken Courses in Engineering on Professional Skills 

Scale 12 

The Emphasis of all Taken Courses in Engineering on Problem Solving Skills 

Scale 13 

The Emphasis of EDSGN 100 (IE 466) on Some Topics in Engineering 

Scale 14 

The Emphasis of EDSGN 100 (IE 466) on Professional Skills 

Scale 15 

The Emphasis of EDSGN 100 (IE 466) on Problem Solving Skills 

Table 1. E2020 survey scales selected for the assessment 


In addition to the selected survey questions from the E2020 survey, one open-ended question 
was added to ask students which course activities they thought improved their awareness of GSEE 
issues. The idea behind inclusion of this question was an affirmation of the source of the impact; in 
other words, have the students perceived the embedded archeology curriculum as the main source 
to increase their awareness on GSEE concepts? 

The data collection and subsequent analysis served to answer two questions. First, in comparison 
to product dissection activities, how did performance of the product archeology curriculum compare 
in gains of learning in GSEE issues as perceived by students? This question was investigated as part of 
the data collection at the first year course (EDSGN 100). Second, in comparison to other engineering 
courses’ impact, how did a specific course compare in enhancing gains as perceived by students? 
This question was answered using the statistical analysis of the paired scales as noted above. 


RESULTS 


Validity of our scales has been already checked in preliminary work (as part of NSF project 
#0550608 activities); however, in order to account for potential population differences we repeated 
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it for our specific project. Reliability can be estimated by four methods: (1) the re-test method, (2) the 
alternative-form method, (3) the split-halves method, and (4) the internal consistency method. The 
internal consistency method is the most suitable method for estimating reliability in our case as 
this method does not require splitting or repeating of items. Cronbach’s alpha is the most popular 
reliability estimate (Carmines and Zeller, 1979). Cronbach alpha scores for our scales were found to 
be at acceptable levels, with 12 out of 15 scores greater than 0.8 (see Table 3, second column). 

Data Analysis for EDSGN 100 

Because EDSGN 100 is the first engineering course for most of our students, we did not expect 
to see identifiable differences across the scale pairs. Accordingly, we opted to compare student 
perceptions against EDSGN 100 course sections where only product dissection was implemented 
(Sections A and B), where dissection and global design were implemented (Section C) and where 
product archeology was embedded in addition to product dissection (Section D). Because the in¬ 
coming quality of students could be salient in the first year, we analyzed the potential differences 
across course sections. 

Data in Table 2 presents the Scale 1 results across gender and class standing level. As can be seen 
in the table, a total of 92 students participated in the EDSGN 100 focused data collection. General 
Linear Modeling (GLM) and Bonferroni multiple comparisons were used to understand the signifi¬ 
cance level of differences across sections due to curricular differences. GLM is used to test statisti¬ 
cal significance of differences. The normality and the existence of equal variances should be tested 
in order to apply the GLM. Normality assumption can be verified by looking at the data histogram 
and normal probability plot. For all scales, normality and variance equivalence are tested although 
we only show detailed results for Scale 1 in the Appendix A. Figure A-la and A-lb (in the Appendix 
A) show the histogram and the normal probability plots for Scale 1. The histogram shows that the 
data is skewed to the left, and the normal probability plot shows some deviation from normality; 
accordingly, a data transformation is done, and normality is rechecked. By applying the Johnson 
transformation, Scale 1 can be transformed to a normally distributed variable. Figure A-2 shows the 
Johnson transformation completed in Minitab. Figure A-3a and A-3b show the histogram and the 
normal probability plots, respectively, for Scale 1 after the transformation. In order to check the as¬ 
sumption of equal error variance, Levene’s test has been implemented. 

Based on these affirmations (normality and equal variances), GLM is implemented to analyze the 
significance of gender, section (intervention; product dissection only or product archeology), year (class 
standing), and their interactions having the scales as the performance measures. Table 3 presents these 
data, where only two scales had statistical significant differences across sections, Scale 1 (Applying 
math and science) and Scale 13 (The Emphasis of EDSGN 100 on Some Topics in Engineering). 
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Section Gender 

Year 

Mean 

Std. Deviation 

N 

A Males 

Freshman 

2.982 

1.0807 

14 


Total 

2.982 

1.0807 

14 

Females 

Freshman 

2.929 

.7734 

7 


Total 

2.929 

.7734 

7 

Total 

Freshman 

2.964 

.9692 

21 


Total 

2.964 

.9692 

21 

B Males 

Freshman 

2.804 

.9208 

14 


Sophomore 

2.417 

.7217 

3 


Junior 

3.500 

.7071 

2 


Total 

2.816 

.8813 

19 

Females 

Freshman 

3.292 

.7144 

6 


Total 

3.292 

.7144 

6 

Total 

Freshman 

2.950 

.8758 

20 


Sophomore 

2.417 

.7217 

3 


Junior 

3.500 

.7071 

2 


Total 

2.930 

.8555 

25 

C Males 

Freshman 

3.346 

.8135 

13 


Sophomore 

3.750 

.4330 

3 


Total 

3.422 

.7622 

16 

Females 

Freshman 

3.250 


1 


Total 

3.250 


1 

Total 

Freshman 

3.339 

.7821 

14 


Sophomore 

3.750 

.4330 

3 


Total 

3.412 

.7392 

17 

D Males 

Freshman 

3.650 

.6559 

20 


Sophomore 

3.333 

.8036 

3 


Total 

3.609 

.6650 

23 

Females 

Freshman 

3.600 

.7202 

5 


Sophomore 

4.500 


1 


Total 

3.750 

.7416 

6 

Total 

Freshman 

3.640 

.6538 

25 


Sophomore 

3.625 

.8780 

4 


Total 

3.638 

.6701 

29 

Total Males 

Freshman 

3.238 

.9076 

61 


Sophomore 

3.167 

.8292 

9 


Junior 

3.500 

.7071 

2 


Total 

3.236 

.8850 

72 

Females 

Freshman 

3.237 

.7287 

19 


Sophomore 

4.500 


1 


Total 

3.300 

.7635 

20 

Total 

Freshman 

3.237 

.8641 

80 


Sophomore 

3.300 

.8882 

10 


Junior 

3.500 

.7071 

2 


Total 

3.250 

.8565 

92 

Table 2. Descriptive statistics for scale 1: applying math and science. 
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# 

Cronbach’s Alpha 

Gender 

Section 

Year 

Gender*Section 

Gender*Year 

Section*Year 

Scale 1 

0.840 

.324 

.036 

.432 

.761 

.177 

.520 

Scale 2 

0.911 

.471 

.072 

.225 

.222 

.208 

.680 

Scale 3 

0.832 

.496 

.749 

.164 

.295 

.414 

.776 

Scale 4 

0.883 

.807 

.297 

.315 

.306 

.982 

.876 

Scale 5 

0.915 

.139 

.728 

.699 

.505 

.299 

.829 

Scale 6 

0.854 

.565 

.974 

.358 

.257 

.711 

.925 

Scale 7 

0.432 

.309 

.885 

.799 

.473 

.739 

.883 

Scale 8 

0.797 

.686 

.465 

.542 

.737 

.712 

.438 

Scale 9 

0.636 

.326 

.132 

.805 

.463 

.733 

.789 

Scale 10 

0.859 

.595 

.075 

.747 

.329 

.893 

.558 

Scale 11 

0.820 

.733 

.484 

.833 

.494 

.914 

.536 

Scale 12 

0.822 

.651 

.174 

.424 

.216 

1.000 

.773 

Scale 13 

0.879 

.096 

.028 

.141 

.524 

.621 

.278 

Scale 14 

0.877 

.775 

.966 

.130 

.103 

.389 

.153 

Scale 15 

0.842 

.540 

.542 

.335 

.512 

.662 

.467 


Table 3. Cronbach’s alpha and p-values 


The relevant items of the scales with significant differences are discussed next. For Scale 1, Section 
D was significantly different than Sections A and B (seen in Appendix A, Table 10); for Scale 13, 
Section C was significantly better than the others (A, B, and D). 

Scale 1. Applying math & science: 

1.1. Math to engineering problems 

1.2. The physical sciences to engineering problems 

1.3. Computer tools and applications to engineering problems 

1.4. Life sciences to engineering problems 

Scale 13. How much did EDSGN100 alone emphasize each of the following topics 
in engineering: 

13.1 Ethical issues in engineering practice 

13.2 The importance of life-long learning 

13.3 Examining my beliefs and values and how they affect my ethical decisions 

13.4 The value of gender, racial/ethnic, or cultural diversity in engineering 

13.5 Creativity and innovation 
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13.6 Current workforce and economic trends 

13.7 Emerging engineering technologies 

13.8 How theories are used in engineering practice 

With relevance to Question 1 (In comparison to product dissection activities, how did performance 
of the product archeology curriculum compare in gains of learning in GSEE issues?), a deeper look 
at Scale 15 was required. Although as an overall scale, the course sections were not significantly dif¬ 
ferent for this scale, Item 1 directly related to GSEE issues (see below), we compared performance 
just for that. However, the comparison of Item15.1 revealed no significant results; in other words, 
Sections A, B, C, and D performed equivalently. 

Scale 15. How much did EDSGN100 alone emphasize each of the following problem 
solving skills: 

15.1 Understanding how an engineering solution can be shaped by environmental, cultural, eco¬ 
nomic considerations. 

Despite the seemingly equivalent performance across section, we also reviewed the responses to 
the open-ended questions: Please indicate EDSGN 100 class activities with which you were engaged 
in/thought of the following: a) How global context influences design, b) How economic context 
influences design, c) How environmental context influences design, and d) How societal context 
influences design. 

Open-ended responses were coded by a senior doctoral student who was given the course syllabi 
of all course sections. This student reviewed all entries and matched the wording from the relevant 
syllabus to ensure that responses coming from the same section’s students pointed to course items 
consistently. For example, design project 2 also meant the industry-sponsored project, but this was 
impossible to discern without the course syllabi. This graduate student did not know what interven¬ 
tion was implemented in which section; at the same time, when statistical analysis and coding results 
were discussed, project investigators did not know which section was which as they were reported 
as Sections A-D. After the analyses were completed only they were able to match the interventions 
to the statistical results. The responses to these open-ended questions are provided in Table 4. The 
entries in the table not only reflect the course features cited for their impact on enhancing GSEE 
awareness, but also a percentage as calculated by the cumulative number of responses divided by 
the number of students in the course section. Note that an observation of more than 100% is pos¬ 
sible as students were not limited to indicate only one course feature. The percentage was preferred 
in place of counts as the class sizes were different. 

Upon review of Table 4, it is clearly seen that Section D, in which product archeology curriculum 
is embedded, has resulted in a higher percentage of students in identifying course features with 
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#of 


#of 


#of 


#of 


Class Activity 

Res. 

Class Activity 

Res. 

Class Activity 

Res. 

Class Activity 

Res. 


Fire Extinguisher 
Redesign Mini 
Project 

4 

Fire Extinguisher 
Redesign Mini Project 

3 

In Class Discussion 

1 

Product Archeology 

20 


Product Design 

2 

Product Design 

2 

Power Point 

1 

Industry Sponsored 

13 

* 

0) 

Challenges 


Challenges 


Presentations 

Design Project 

o 

Practice Problem of 

A 

Practice Problem of 

o 

Team Work 

1 

Customer Needs 

1 


Water Resource 


Water Resource 

z 

Assessment 

o 

3 

Readings 

1 

Dissection Project 

2 

Global Design 
Project 

5 

Green Design 

3 


Geographic Region 
of the Product 

2 

EMS Modeling 

2 

Projects 

2 



Marketing 

1 

Marketing 

2 



CAD 

1 

% 

SECTION A: 67% 

SECTION B: 52% 

SECTION C: 59% 

SECTION D: 128% 




Dissection Project 

8 



Product Archeology 

17 


Dissection Project 

11 

Industry Sponsored 




Industry Sponsored 
Design Project 

16 


Fire Extinguisher 


Design Project 

1 

Dissection Project 

5 

Customer Needs 

1 


Redesign Mini 

2 





Assessment 

C 

o 

Project 








U 

o 

s 

o 

Problem Solving 

1 

Fire Extinguisher 
Redesign Mini Project 

1 



Camera Dissection 

1 

o 

o 

qq 

EMS Modeling 

1 

EMS Modeling 

1 






Practice Problem of 

1 

Product Dissection 

1 

Global Design 

2 

CAD 

1 


Water Resource 

Project 



Cost Calculations 

1 

Cost Calculations 

3 






Market Competition 

1 

Marketing 

1 

In Class Discussion 

1 

Budget 

1 

% 

SECTION A: 86% 

SECTION B: 64% 

SECTION C: 47% 

SECTION D: 128% 








Product Archeology 

12 


Dissection Project 

7 

Dissection Project 

4 



Industry Sponsored 
Design Project 

21 

* 

Fire Extinguisher 
Redesign Mini 
Project 

5 

Industry Sponsored 
Design Project 

1 

Dissection Project 


Geographic 

Locations 

1 

c 

o 

EMS Modeling 

1 

EMS Modeling 

1 


3 

CAD 

1 

U 

13 

Practice Problem of 

1 

Practice Problem of 

o 



Camera Dissection 

1 

c 

0) 

Water Resource 

Water Resource 

9 



£ 

3 

o 

Green Design 

1 

Product Dissection 

1 









Global Design 




§ 

qq 

Energy Saving 

1 

Fire Extinguisher 
Redesign Mini Project 

1 

Project 





Product Design 
Challenges 

2 

Design Problems 

1 


1 

Green Design 





Lab2-Noise and 







Readings 

1 

Battery Life 
Measurement 

3 

In Class Discussion 

2 


7 


Table 4 Open-ended question summary 
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( Continued ) 



Class Activity 

#of 

Res. 

Class Activity 

# of 
Res. 

Class Activity 

#of 

Res. 

Class Activity 

# of 
Res. 

% 

SECTION A: 90% 


SECTION B: 64% 


SECTION C: 35% 

SECTION D: 148% 








Product Archeology 

15 


Dissection Project 

9 

Dissection Project 

9 



Industry Sponsored 
Design Project 

14 


Fire Extinguisher 
Redesign Mini 
Project 

3 

Industry Sponsored 
Design Project 

1 

Dissection Project 

4 

Japanese Culture 

1 

>< 

£ 

a 

o 

Customer Needs 

1 

EMS Modeling 

1 



Customer Needs 
Assessment 

1 

U 

'cS 

Team Work 

1 

Practice Problem of 

1 





O 

t o 

Water Resource 

Global Design 

2 




Product Design 
Challenges 

4 

Design Problems 

2 

Project 


CAD 

2 




Women in 

Engineering 

1 

In Class 

Presentation 





Readings 

1 

Customer Needs 
Analysis 

2 

1 

Green Design 

2 

% 

SECTION A: 90% 


SECTION B: 68% 


SECTION C: 41% 

SECTION D: 121% 


Table 4 Open-ended question summary 


relevance to GSEE activities. Given that, by and large, all these 92 students have similar qualities as 
identified by the mostly equivalent scale values, it is very encouraging to see that product archeol¬ 
ogy curriculum was able to garner much higher awareness of GSEE issues. 

Data Analysis for IE 466 

Across the two data collection years a total of 57 valid observations were achieved. Using these 
data, this part of the analysis concentrated on Question 2: “In comparison to other engineering 
courses’ impact how did IE 466 compare in enhancing GSEE gains as perceived by students?” This 
question was answered using the statistical analysis of the paired scales as noted in Section 4. 
Descriptive statistics and significance test tables are provided in Tables 5 and 6 for Scales 10 and 
13, and in Tables 7 and 8 for Scales 12 and 15. 

A review of Table 5 shows that on all scale items students rated their experience in IE 466 to be 
better in comparison to all other engineering courses. The differences are also evaluated as shown 
in Table 6. It is seen that Items 5-8 revealed significant differences. Item 6 focuses on “current work¬ 
force and economic trends” and, thus, is related to GSEE issues. 
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Std. 

Std. Error 



Mean 

N 

Deviation 

Mean 

Ethical issues in engineering practice. 

Scale 10.1 

3.39 

57 

1.114 

.148 

Scale 13.1 

3.51 

57 

.889 

.118 

The importance of life-long learning. 

Scalel0.2 

3.67 

57 

1.006 

.133 

Scalel3.2 

3.68 

57 

.929 

.123 

Examining my beliefs and values and how they affect my 

Scale 10.3 

3.04 

57 

1.133 

.150 

ethical decisions. 

Scale 13.3 

3.32 

57 

1.072 

.142 

The value of gender, racial/ethnic, or cultural diversity in 

Scalel0.4 

2.95 

57 

1.260 

.167 

engineering. 

Scale 13.4 

3.11 

57 

1.030 

.136 

Creativity and innovation. 

Scalel0.5 

4.14 

57 

.915 

.121 

Scalel3.5 

4.67 

57 

.607 

.080 

Current workforce and economic trends (globalization, 

Scalel0.6 

3.82 

57 

.928 

.123 

outsourcing, etc.). 

Scalel3.6 

4.21 

57 

.881 

.117 


Scalel0.7 

3.70 

57 

.981 

.130 

Emerging engineering technologies. 

Scale 13.7 

4.05 

57 

.833 

.110 

How theories are used in engineering practice. 

Scale 10.8 

3.74 

57 

.917 

.121 

Scale 13.8 

4.09 

57 

.912 

.121 


Table 5. Paired scales statistics , scales 10 & 13. 




Mean 

Paired Differe 

Std. 

Std. Error 

Deviation Mean 

nces 

95% 

Confidence 
Interval of the 
Difference 

t 

Sig. 

df (2-tailed) 

Lower 

Upper 

Pair 1 

Scale 10.1 - Scale 13.1 

-.123 

1.415 

.187 

-.498 

.253 

-.655 

56 

.515 

Pair 2 

Scalel0.2 - Scalel3.2 

-.018 

.896 

.119 

-.255 

.220 

-.148 

56 

.883 

Pair 3 

Scalel0.3 - Scalel3.3 

-.281 

1.221 

.162 

-.605 

.043 

-1.735 

56 

.088 

Pair 4 

Scalel0.4 - Scalel3.4 

-.158 

1.222 

.162 

-.482 

.166 

-.976 

56 

.333 

Pair 5 

Scalel0.5 - Scalel3.5 

-.526 

.947 

.125 

-.778 

-.275 

-4.196 

56 

.000 

Pair 6 

Scalel0.6 - Scalel3.6 

-.386 

1.098 

.145 

-.677 

-.095 

-2.654 

56 

.010 

Pair 7 

Scalel0.7 - Scalel3.7 

-.351 

1.077 

.143 

-.637 

-.065 

-2.459 

56 

.017 

Pair 8 

Scale 10.8 - Scalel3.8 

-.351 

1.246 

.165 

-.682 

-.020 

-2.126 

56 

.038 


Table 6. Significant differences (scale 10-scale 13). 
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Scale comparison 12 vs. 15 focuses on Problem Solving Skills in general, emphasizing how knowl¬ 
edge from other fields can be used in solving engineering problems. Relevant to our focus and the 
embedment of the product archeology curriculum into IE 466, the first item intends to compare 
how the course in comparison to all other engineering courses enhances students’ “ Understanding 
on how an engineering solution can be shaped by environmental, cultural, economic, and other 
considerations As it can be followed in Tables 7 and 8, on this specific item along with Items 2 and 
4, students rated the IE 466 course to have a higher rating in comparison to all other engineering 
courses they have taken. The significance of these results is also confirmed in Table 8. 

Overall, across the scale pairs analyzed, the perceived impact of all other engineering courses 
students have taken versus the product archeology embedded course (IE 466) alone, the results 
point to the effectiveness in contributions to one carefully designed course or set of product ar¬ 
chaeology activities. Considering that the first item in Scale pair 12 and 15 is “ Understanding how an 
engineering solution can be shaped by environmental, cultural, economic, and other considerations”, 
overall it is concluded that the efforts presented are in the right direction. 

In an effort to understand how much impact we can expect with the product archeology inter¬ 
vention at different class standing levels (first-year to senior), we have compared the paired-scale 
differences across courses using t-tests. The results are presented in Table 11 of Appendix B. Other 
than only few items (i.e., 11-14.1 (professional skills); 11-14.3 (leadership skills); 12-15.1 (systems thinking) 
and 12-15.5 (defining a design problem)), the positive impact of the product archeology curriculum 




Mean 

N 

Std. 

Deviation 

Std. Error 
Mean 

Understanding how an engineering solution can be shaped by 
environmental, cultural, economic, and other considerations 

Scalel2.1 

3.46 

57 

.983 

.130 

Scale 15.1 

3.98 

57 

.876 

.116 

Understanding how non-engineering fields can help solve 

Scalel2.2 

3.04 

57 

1.117 

.148 

engineering problems 

Scalel5.2 

3.75 

57 

.872 

.115 

Systems thinking 

Scalel2.3 

3.32 

57 

1.121 

.148 


Scale 15.3 

3.11 

57 

1.205 

.160 

Applying knowledge from other fields to solve an engineering 

Scalel2.4 

3.13 

56 

1.010 

.135 

problem 

Scale 15.4 

3.55 

56 

.829 

.111 

Defining a design problem 

Scalel2.5 

3.98 

57 

.896 

.119 


Scalel5.5 

4.11 

57 

.838 

.111 

Generating and evaluating ideas about how to solve an 

Scalel2.6 

4.09 

57 

.830 

.110 

engineering problem 

Scalel5.6 

4.18 

57 

.848 

.112 


Table 7. Paired scales statistics, scales 12 & 15. 
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Paired Differences 

95% Confidence 




Mean 

Std. 

Deviation 

Std. 

Error 

Mean 

Interval of the 
Difference 

t 

df 

Sig. 

(2-tailed) 

Lower 

Upper 

Pair 1 

Scalel2.1 - Scalel5.1 

-.526 

.984 

.130 

-.787 

-.265 

-4.039 

56 

.000 

Pair 2 

Scalel2.2 - Scalel5.2 

-.719 

1.013 

.134 

-.988 

-.450 

-5.359 

56 

.000 

Pair 3 

Scalel2.3 - Scalel5.3 

.211 

1.191 

.158 

-.106 

.527 

1.334 

56 

.188 

Pair 4 

Scale 12.4 - Scale 15.4 

-.429 

.931 

.124 

-.678 

-.179 

-3.443 

55 

.001 

Pair 5 

Scalel2.5 - Scalel5.5 

-.123 

1.019 

.135 

-.393 

.148 

-.910 

56 

.367 

Pair 6 

Scalel2.6 - Scalel5.6 

-.088 

1.106 

.147 

-.381 

.206 

-.599 

56 

.552 


Table 8. Significant differences (scale 12-scale 15). 


in comparison to students’ all other engineering courses significantly grew (as per students’ percep¬ 
tion data). These results also point to the effectiveness of the curriculum. 

We note several limitations in our assessment. First of all, our results are mostly based on self- 
reported student perceptions, and we are not testing content knowledge or learning directly. In addi¬ 
tion, we are only testing two courses to embed the product archeology curriculum, and although we 
have four course sections in which we tested the curriculum at the first-year course (EDSGN 100), for 
IE 466 multiple sections were not offered. In relation to the latter point, the limitation due to having 
one section precludes us from being definitive with the conclusions about the GSEE activities, i.e., 
we cannot definitively say that the GSEE activity is what makes IE 466 different—it could just be the 
overall topical coverage of IE 466 (beyond the mini curriculum). Since we do not have data for this 
course prior to introducing the GSEE activity, it is impossible to affirm GSEE activities’ sole contribu¬ 
tion. Current data collections are underway to eliminate the limitations we are citing here. 


DISCUSSION AND CONCLUSIONS 

In this paper, using a validated survey we have investigated the effectiveness of the product 
archeology curriculum in enhancing students’ awareness on GSEE issues. Assessment was done in 
a first-year course as well as in a senior-level technical elective. With the first-year level data collec¬ 
tion, we have sought to understand if product archeology curriculum can more effectively introduce 
the GSEE issues in comparison to frequently used product dissection and redesign projects. Ac¬ 
cordingly, four sections of the EDSGN 100 course were included in the study, where two sections 
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(A and B) included product dissection projects and an industry-sponsored project, one section (C) 
included a product dissection project and a global design project, and one section (D) included a 
product archeology project and an industry-sponsored project. The comparisons across sections 
first ensured that students were overall at similar levels as assessed using the 15 scales. A review 
of the responses to the open-ended questions revealed that students in the course section, which 
embedded product archeology curriculum, had a much higher level of identification of course fea¬ 
tures that impacted their awareness on GSEE issues. 

The data collection and subsequent analysis completed for the IE 466 course revealed that 
across half of the item pairs for Scales 10 and 13 as well as Scales 12 and 15, the product archeol¬ 
ogy curriculum positively and significantly impacted students’ perception of their learning in areas 
relevant to GSEE issues. 

Overall, embedding product archeology is found to be effective in both the first year (as com¬ 
pared to the other sections of the course that did not use the product archeology) as well as the 
senior year (as compared to all other engineering courses). Given the fact that in both courses, the 
implementation activities took no more than 5 hours of actual in class time, we assert that product 
archeology is an effective and efficient way of increasing awareness of GSEE concepts in engineer¬ 
ing students based on the assessment results. These results indicate that the “product archeology” 
can be an effective pedagogy to create the desired effect to address ABET’s Outcome h. However, 
as acknowledged further data collection is necessary in other courses, and we are developing as¬ 
sessment methods to use content knowledge/learning data to draw conclusive evidence. 
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APPENDIX A 


Statistical procedures followed for each scale across data collection years. 
1. Normality test is done. 


Histogram 



Scald 


a) 


Normal Q-Q Plot of Scald 



Figure A-1. a) Histogram, b) Normal probability plot of scale 1. 
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Johnson Transformation for Scalel 


Probability Plot for Original Data Select a Transformation 




Probability Plot for Transformed Data 



P-Value for Best Fit: 0.0579488 
Z for Best Fit: 0.9 
Best Transformation Type: SB 
Transformation function equals 

-1.63970 + 1.87726 * Ln( ( X + 1.51595 ) / ( 5.42585 - X )) 


Figure A-2. Johnson transformation for scale 7. 


If the data is not distributed normally, data transformation was done. By applying Johnson 
transformation, scale 1 can be transformed to a normally distributed variable. Figure A-2 shows the 
Johnson transformation completed in Minitab. 

Figure A-3.a and A-3.b show the histogram and the normal probability plots, respectively, for 
scale 1 after the transformation. 

In order to check the assumption of equal error variance, Levene’s test has been implemented 
as shown in Table 8. We fail to reject the null hypothesis (equal error variance) because the 
p = 0.770 > 0.05. 


Dependent Variable:Scale1 T 
F dfl df2 Sig. 

.675 12 79 .770 


Table 8. Levene’s test for scale 1 (Levene’s test of equality of error variances) 
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Histogram 


Normal Q-Q Plot of Scale1_T 



Figure A-3.a Histogram 



-i-1-1-1-1-1-r 

• 3 - 2-10123 

Observed Value 


Figure A-3.b Normal Probability Plot of Scale 1 


GLM can be implemented after the confirmation of normality and equal variances. Table 9 
shows the ANOVA table. ANOVA table shows that there are significant differences across sections 
(p=0.036 < 0.05). While the other variables and the interactions are not statistically significant 
(p > 0.05). 

Table 10 shows the Bonferroni multiple comparisons. Using this method, we can identify which 
sections have significantly different means. According to Bonferroni multiple comparisons, section 
A and D, and section B and D are statistically different. 


Source 

Type IV Sum 
of Squares 

Df 

Mean Square 

F 

Sig. 

Corrected Model 

22.736 a 

12 

1.895 

1.561 

.121 

Intercept 

.315 

1 

.315 

.260 

.612 

Section 

10.879 b 

3 

3.626 

2.989 

.036 

Gender 

1.194 b 

1 

1.194 

.984 

.324 

Year 

2.056 b 

2 

1.028 

.847 

.432 

Section * Gender 

1.419 b 

3 

.473 

.390 

.761 

Section * Year 

1.601 b 

2 

.800 

.660 

.520 

Gender * Year 

2.253 b 

1 

2.253 

1.857 

.177 

Section * Gender * Year 

.000 

0 




Error 

95.862 

79 

1.213 



Total 

118.629 

92 




Corrected Total 

118.598 

91 




a . R Squared = .192 (Adjusted R Squared = .069), b . The Type IV testable hypothesis is not unique. 


Table 9. ANOVA output (tests of between-subjects effects). 
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(I) 

Section 

(J) 

Section 

Mean 

Difference 

(I-J) 

Std. 

Error 

Sig. 

95 % Confidence Interval 

Lower 

Bound Upper Bound 

A 

B 

.043 

.3261 

1.000 

-.839 

.926 


C 

-.537 

.3594 

.837 

-1.509 

.436 


D 

-.908* 

.3156 

.031 

-1.763 

-.054 

B 

A 

-.043 

.3261 

1.000 

-.926 

.839 


C 

-.580 

.3463 

.589 

-1.517 

.358 


D 

-.951* 

.3006 

.013 

-1.765 

-.138 

C 

A 

.537 

.3594 

.837 

-.436 

1.509 


B 

.580 

.3463 

.589 

-.358 

1.517 


D 

-.372 

.3365 

1.000 

-1.282 

.539 

D 

A 

.908* 

.3156 

.031 

.054 

1.763 


B 

.951* 

.3006 

.013 

.138 

1.765 


C 

.372 

.3365 

1.000 

-.539 

1.282 

Based on observed means. The error term is Mean Square(Error) 
the .05 level. 

= 1.213. *. 

The mean difference 

is significant at 


Table 10. Bonferroni multiple comparisons of scale 1. 
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APPENDIX B. COMPARISON OF FIRST-YEAR STUDENTS TO SENIOR STUDENTS 



Levene’s 


t-test for Equality of Means 


Test for 



95% Confidence 

Equality of 



Interval of the 

Variances 


Sig. Mean Std. Error 

df (2-tailed) Difference Difference 

Difference 

F Sig. 

t 

Lower Upper 


Scalel0.1_13.1 Equal variances 2.517 .115 -5.358 




Unequal variances 



-5.514 

Scal210.2_ 

_13.2 

Equal variances 

8.002 

.005 

-1.934 



Unequal variances 



-2.059 

Scale 10.3_ 

_13.3 

Equal variances 

2.646 

.106 

-3.377 



Unequal variances 



-3.461 

Scale 10.4_ 

_13.4 

Equal variances 

7.180 

.008 

-2.709 



Unequal variances 



-2.821 

Scale 10.5_ 

_13.5 

Equal variances 

1.936 

.166 

-4.295 



Unequal variances 



-4.572 

Scale 10.6_ 

.13.6 

Equal variances 

1.629 

.204 

-5.653 



Unequal variances 



-5.810 

Scale 10.7_ 

.13.7 

Equal variances 

10.357 

.002 

-2.813 



Unequal variances 



-2.948 

Scale 10.8_ 

.13.8 

Equal variances 

1.641 

.202 

-3.527 



Unequal variances 



-3.611 

Scalell.l. 

.14.1 

Equal variances 

1.071 

.303 

.159 



Unequal variances 



.156 

Scale 11.2_ 

.14.2 

Equal variances 

3.010 

.085 

-3.204 



Unequal variances 



-3.317 

Scale 11.3_ 

.14.3 

Equal variances 

.479 

.490 

-.661 



Unequal variances 



-.675 

Scale 11.4_ 

.14.4 

Equal variances 

.769 

.382 

-1.900 



Unequal variances 



-2.022 

Scale 11.5_ 

.14.5 

Equal variances 

.012 

.913 

-1.842 



Unequal variances 



-1.860 

Scalel2.1_ 

.15.1 

Equal variances 

4.308 

.040 

-2.846 



Unequal variances 



-2.924 

Scale 12.2_ 

.15.2 

Equal variances 

4.880 

.029 

-4.855 



Unequal variances 



-5.120 

Scalel2.3_ 

.15.3 

Equal variances 

.009 

.926 

.407 



Unequal variances 



.407 

Scale 12.4_ 

.15.4 

Equal variances 

1.688 

.196 

-2.954 



Unequal variances 



-3.105 

Scale 12.5_ 

.15.5 

Equal variances 

.235 

.629 

-1.511 


134 

.000 

-.92500 

.17263 

-1.26644 

-.58356 

128.960 

.000 

-.92500 

.16775 

-1.25689 

-.59311 

134 

.055 

-.40357 

.20865 

-.81624 

.00910 

133.995 

.041 

-.40357 

.19604 

-.79130 

-.01584 

134 

.001 

-.69107 

.20463 

-1.09579 

-.28636 

127.684 

.001 

-.69107 

.19968 

-1.08617 

-.29597 

134 

.008 

-.56964 

.21031 

-.98561 

-.15368 

131.890 

.006 

-.56964 

.20192 

-.96907 

-.17022 

134 

.000 

-.58571 

.13638 

-.85546 

-.31597 

133.992 

.000 

-.58571 

.12811 

-.83909 

-.33233 

134 

.000 

-.96429 

.17057 

-1.30165 

-.62692 

128.542 

.000 

-.96429 

.16598 

-1.29269 

-.63588 

134 

.006 

-.49107 

.17458 

-.83635 

-.14579 

132.938 

.004 

-.49107 

.16659 

-.82057 

-.16157 

134 

.001 

-.60893 

.17267 

-.95043 

-.26742 

127.443 

.000 

-.60893 

.16862 

-.94258 

-.27527 

134 

.874 

.02679 

.16878 

-.30703 

.36060 

112.334 

.876 

.02679 

.17120 

-.31241 

.36598 

134 

.002 

-.50714 

.15826 

-.82015 

-.19413 

130.498 

.001 

-.50714 

.15290 

-.80963 

-.20466 

134 

.510 

-.11786 

.17821 

-.47032 

.23460 

126.282 

.501 

-.11786 

.17464 

-.46346 

.22775 

134 

.060 

-.26607 

.14007 

-.54310 

.01096 

133.994 

.045 

-.26607 

.13159 

-.52634 

-.00580 

134 

.068 

-.30893 

.16774 

-.64069 

.02284 

122.466 

.065 

-.30893 

.16611 

-.63776 

.01990 

134 

.005 

-.47679 

.16755 

-.80817 

-.14540 

128.518 

.004 

-.47679 

.16305 

-.79940 

-.15417 

134 

.000 

-.89464 

.18426 

-1.25908 

-.53021 

133.646 

.000 

-.89464 

.17474 

-1.24026 

-.54902 

134 

.685 

.08571 

.21078 

-.33116 

.50259 

118.469 

.685 

.08571 

.21080 

-.33171 

.50314 

134 

.004 

-.51607 

.17470 

-.86161 

-.17054 

133.347 

.002 

-.51607 

.16618 

-.84476 

-.18738 

134 

.133 

-.23929 

.15840 

-.55257 

.07400 

127.353 

.124 

-.23929 

.15473 

-.54546 

.06689 

134 

.080 

-.27321 

.15463 

-.57904 

.03261 

123.811 

.076 

-.27321 

.15259 

-.57523 

.02880 


Unequal variances 
Scalel2.6_15.6 Equal variances 
Unequal variances 


1.330 .251 


-1.546 

-1.767 

-1.791 


Table 11. Independent samples test. 
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